Papers with automated approaches
REM: Efficient Semi-Automated Real-Time Moderation of Online Forums (2021.acl-demo)
Copied to clipboard
| Challenge: | REM is a tool for the semi-automated real-time moderation of large scale online forums. |
| Approach: | They propose a semi-automated real-time moderation tool for large scale online forums that maximizes the efficiency of manual moderation by targeting only those comments for which human intervention is needed. |
| Outcome: | The proposed method maximizes the efficiency of manual moderation by targeting only those comments for which human intervention is needed, e.g. due to high classification uncertainty. |
CASPER: Bridging Discrete and Continuous Prompt Optimization through Feedback-Guided Gradient Descent (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing pipelines for generative tasks require extensive manual effort and domain expertise to achieve task-optimal performance. |
| Approach: | They propose a framework bridging discrete and continuous prompt optimization through feedback-guided gradient descent in embedding space. |
| Outcome: | The proposed framework bridges discrete and continuous prompt optimization through feedback-guided gradient descent in embedding space. |
Treating Dialogue Quality Evaluation as an Anomaly Detection Problem (2020.lrec-1)
Copied to clipboard
| Challenge: | Dialogue systems for interaction with humans are becoming more popular . the best way to estimate their success is through means of human evaluation . |
| Approach: | They investigate the effectiveness of perceiving dialogue evaluation as an anomaly detection task. |
| Outcome: | The proposed approach is based on four models and shows negative results . the proposed approach could be used in the future to improve human-led dialogue evaluations. |
Accurate and Well-Calibrated ICD Code Assignment Through Attention Over Diverse Label Embeddings (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to assigning ICD codes to clinical text are time-consuming, labor intensive, and error-prone. |
| Approach: | They propose to adapt a Transformer-based model to a longformer model and use it to encode clinical narratives. |
| Outcome: | The proposed approach outperforms current state-of-the-art models in ICD coding with the label embeddings contributing to the good performance. |
Toxicity Detection is NOT all you Need: Measuring the Gaps to Supporting Volunteer Content Moderators through a User-Centric Method (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing efforts to automate content moderation have focused on identifying toxic, offensive, and hateful content . yet, it remains unclear whether improvements have addressed the needs of volunteer content moderators . |
| Approach: | They propose to use a model review to examine the availability of moderators' models to flag violations of various forum rules. |
| Outcome: | The proposed models perform poorly on a significant portion of the rules. |
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Acceptability judgments are the most significant source of data in linguistics . however, there are still many open issues regarding methods for collecting and evaluating them. |
| Approach: | They propose to create a corpus of sentences with acceptability judgments using the same approach and the same steps as the English corpus. |
| Outcome: | The proposed corpus contains almost 10,000 sentences with acceptability judgments. |
AutoTaskEval: Towards Domain-Specific and Fine-Grained Evaluation for LLMs (2026.acl-long)
Copied to clipboard
Qingqing Lyu, Linjuan Wu, Yongliang Shen, Hengwei Liu, Hao Li, Shengpei Jiang, Yin Zhang, Weiming Lu
| Challenge: | Existing automated approaches operate within fixed task schemas and often fail to autonomously discover new evaluation dimensions. |
| Approach: | They propose an automated framework that constructs domain-specific benchmarks directly from unstructured corpora using Bloom’s Taxonomy. |
| Outcome: | The proposed framework uncovers a broader and more fine-grained task space than expert-curated benchmarks while producing high-quality instances that preserve established model-level evaluation trends. |
PDFdigest: an Adaptable Layout-Aware PDF-to-XML Textual Content Extractor for Scientific Articles (L18-1)
Copied to clipboard
| Challenge: | Existing tools to extract structured textual content from PDFs are essential to enable scientific text mining. |
| Approach: | They propose a PDF-to-XML textual content extraction tool that extracts structured textual contents from scientific articles in PDF format. |
| Outcome: | The proposed tool extracts structured textual content from scientific articles in PDF format while preserving both the textual contents and layout details of the input PDF document. |
Reproducibility and Automation of the Appraisal Taxonomy (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for Appraisal annotation are descriptive and lack of data hinders progress . |
| Approach: | They propose to use annotated data to measure the performance of automated Appraisal annotations in a publicly available dataset. |
| Outcome: | The proposed methods show poor agreement at more detailed categories and fair agreement at coarse-level categories. |
Building Semantic Grams of Human Knowledge (2020.lrec-1)
Copied to clipboard
| Challenge: | Word senses are typically defined with textual definitions and put in context via lexical-semantic relations such as synonymy, antonymy, hypernymy, etc. |
| Approach: | They propose a slot-filler structure to define the meaning of words in terms of their prototypical semantic information. |
| Outcome: | The proposed model improves on a semantic similarity task and shows significant improvements over state-of-the-art embeddings. |
On the Effectiveness of Adversarial Robustness for Abuse Mitigation with Counterspeech (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work on automated counterspeech systems focused on synthetic data but rarely looked into how the public deals with abuse. |
| Approach: | They propose to curate a new dataset of abuse and replies from footballers for study of public figure abuse and use it to examine how models can handle adversarial attacks. |
| Outcome: | The proposed model is robust against adversarial attacks across domains and can handle abuse in the real world. |
Paragraph-level Simplification of Medical Texts (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for simplification of medical texts are limited due to jargon and technical content. |
| Approach: | They propose to automate the simplification of medical texts by penalizing decoders for producing "jargon" terms. |
| Outcome: | The proposed method improves on existing heuristics by penalizing the decoder for producing "jargon" terms. |
Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing NLP task definitions for fact-checking cannot refute misinformation as professional fact- checkers do for the majority of claims. |
| Approach: | They compare NLP-based fact-checking with professional fact- checkers . they find that evidence must be sufficient to refute the claim and not leaked . |
| Outcome: | The proposed models fail to meet the criteria for realistic fact-checking . they also fail to satisfy the criteria of leaked evidence . |
“Image, Tell me your story!” Predicting the original meta-context of visual misinformation (2024.emnlp-main)
Copied to clipboard
| Challenge: | a surge of visual misinformation poses a growing threat to our society. |
| Approach: | They propose to use images to contextualize images to establish original meta-context . they use a framework that collects evidence and questions from the open web to ground the image . |
| Outcome: | The proposed approach helps human fact-checkers identify misinformation . it uses image contextualization to establish the original meta-context of the image . |
A Lightweight Method to Generate Unanswerable Questions in English (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to build robust question answering models are too complex . antonym and entity swaps on answerable questions are used to build models . |
| Approach: | They propose a method for performing antonym and entity swaps on unanswerable questions. |
| Outcome: | The proposed method outperforms the previous state-of-the-art and has higher human-judged relatedness and readability. |
SyllabusQA: A Course Logistics Question Answering Dataset (2024.acl-long)
Copied to clipboard
| Challenge: | We introduce Fact-QA, an LLM-based evaluation metric to evaluate the factuality of predicted answers. |
| Approach: | They propose to use an open-source dataset to analyze logistics-related question-answer pairs in a logistics-based course. |
| Outcome: | The proposed approach performs close to humans on traditional metrics of textual similarity, but there is a significant gap between them and humans in terms of fact precision. |
Cognitive Linguistic Identity Fusion Score (CLIFS): A Scalable Cognition‐Informed Approach to Quantifying Identity Fusion from Text (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for measuring identity fusion are limited and require controlled surveys or direct field contact. |
| Approach: | They propose a new metric that integrates cognitive linguistics with large language models to measure identity fusion. |
| Outcome: | The proposed metric outperforms existing methods and human annotations in violence risk assessment. |
BAGELS: Benchmarking the Automated Generation and Extraction of Limitations from Scholarly Text (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of scientific publications have limitations as a source of uncertainty. |
| Approach: | They propose a computational architecture for extracting and generating limitations from scholarly papers using a novel Retrieval Augmented Generation technique. |
| Outcome: | The proposed architecture extracts limitations from ACL, NeurIPS, and PeerJ papers and supplementes them with external reviews. |
PhoniTale: Phonologically Grounded Mnemonic Generation for Typologically Distant Language Pairs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Vocabulary acquisition is a challenge for second-language learners when learning typologically distant languages such as English and Korean, where phonological and structural mismatches complicate vocabulary learning. |
| Approach: | They propose a cross-lingual mnemonic generation system that performs IPA-based phonological adaptation and syllable-aware alignment to retrieve L1 keyword sequence and uses LLMs to generate verbal cues. |
| Outcome: | The proposed system outperforms human-written and automated mnemonics in a short-term recall test with human participants and achieves quality comparable to human-writing mnms. |
CheXalign: Preference fine-tuning in chest X-ray interpretation models without human feedback (2025.acl-long)
Copied to clipboard
Dennis Hein, Zhihong Chen, Sophie Ostmeier, Justin Xu, Maya Varma, Eduardo Pontes Reis, Arne Edward Michalson Md, Christian Bluethgen, Hyun Joo Shin, Curtis Langlotz, Akshay S Chaudhari
| Challenge: | Radiologists are a crucial role in translating medical images into actionable reports . however, the field faces staffing shortages and increasing workloads . |
| Approach: | They propose an automated pipeline for preference feedback focusing on chest X-ray radiology report generation (RRG) method leverages publicly available datasets containing pairs of images and radiologist-written reference reports with reference-based metrics, or Judges. |
| Outcome: | The proposed pipeline achieves state-of-the-art CheXbert scores on the MIMIC-CXR dataset while on average maintaining robust performance across six additional image perception and reasoning tasks. |
Efficient Process Reward Modeling via Contrastive Mutual Information (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to verify intermediate reasoning steps require human annotators to assign reward scores to each reasoning step, which is labor-intensive and costly. |
| Approach: | They propose a method that leverages the model's internal probability to infer step-level supervision while significantly reducing the computational burden of annotating dataset. |
| Outcome: | The proposed method reduces dataset construction time by 84% and token generation by 98% compared to MC estimation, while achieving higher accuracy on process-level evaluations and mathematical reasoning benchmarks. |